Conditional branch in machine learning object detection

ABSTRACT

Apparatus, methods, systems, and instructions stored on computer-readable medium are presented for performing classification. A hardware engine may be configurable to implement a plurality of cascade classifiers comprising at least a first cascade classifier and a second cascade classifier, each including one or more stages. The first cascade classifier may be configured to perform a classification different from a classification performed by the second cascade classifier. The hardware engine may be capable of conditionally branching from the first cascade classifier to the second cascade classifier based on an outcome condition associated with evaluating of at least one stage of the first cascade classifier.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/641,151, filed Mar. 9, 2018, entitled “Conditional Branch andAccumulate Across Stages in Machine Learning Object Detection” which isincorporated herein by reference. This application is related toco-pending U.S. Patent Application No. _____ titled “Accumulate AcrossStages in Machine Learning Object Detection,” filed Aug. 31, 2018, andhaving Attorney Docket No. 093495-1083556 (181258U2), the entirety ofwhich is hereby incorporated by reference.

BACKGROUND

Aspects of the disclosure relate to machine learning and, in specificembodiments, to improvements in the use of machine learning in visualobject detection. Visually based object detection, such as facedetection, based on machine learning has tremendous potential for a widerange of interesting and useful applications. By tapping into thecapabilities of machine learning, computers can be trained to detectobjects in images in powerful and scalable ways. However, suchmachine-based object detection can be computationally expensive, whichcan translate to high demands on battery, processing, and memoryresources. These issues become particularly acute in mobile, wireless,wearable, and/or other resource-limited environments. While existingmachine learning techniques have achieved certain gains in efficiency,the performance of practical system for machine learning based objectdetection leaves much to be desired for many applications.

For example, in the realm of multiple classifiers, U.S. PatentPublication No. 2012/0120269 ('269 publication) describes a technique inwhich “[m]ultiple focus-specific classifier chains are applied,including a first chain trained to match significantly out of focusfaces, and a second chain trained to match slightly out of focus faces.”In a more detailed example, the '269 publication describes three cascadeclassifiers: (1) a sharply focused classifier, (2) a slightlyout-of-focus classifier, and (3) a significantly out-of-focusclassifier. See '269 publication, paragraphs [0141]-[0146]. Some or allof these cascade classifiers may be executed at once, but they appear tobe executed independently of one another. See '269 publication, [0143].The '269 publication fails to disclose any conditional branching fromone cascade classifier to another cascade classifier.

As further examples, in the realm of accumulated values, U.S. Pat. No.8,805,081 ('081 patent) describes a method of “performing, by an imageprocessing device, object detection in an image comprising: performingone or more tests of a test sequence for detection of a first object onpixels [sic] values of a plurality of at least partially overlappingsub-regions (310, 312, 314) of a first search window (108); generating acumulative score based on results of said one or more tests on saidplurality of subregions; comparing said cumulative score with athreshold value; and based on said comparison, selectively performingone or more of said tests of said test sequence on at least one furthersub-region of said first search window, said at least one furthersub-region at least partially overlapping each of said plurality ofsub-regions.”'081 patent, Abstract. However, the “cumulative score” (CA)disclosed in the '081 patent is merely an indicator of the depth reachedin a cascade, and the cumulative score is not accumulated across stages.A numeric value obtained from evaluation of a first stage is notaccumulated with a second numeric value obtained from evaluation of asecond stage to generate an accumulated value.

U.S. Patent Publication No. 2006/0257017 ('017 publication) describes aclassification method that includes “accessing an image to be analyzedfor the presence of a predefined object, processing the image using aplurality of evaluation stages, generating a plurality of scores forrespective ones of the evaluation stages responsive to the processing,wherein the scores are indicative of a likelihood of a candidate regionof the image comprising the object, accumulating the scores from theevaluation stages to provide a cumulative score, and using thecumulative score, determining whether the candidate region comprises theobject.” '017 publication, Abstract. However, the technique described inthe '017 publication accumulates scores from the evaluation of stageswithout accessing any value to determine whether to accumulate. Forexample, “the likelihood scores of the current stage 34 and sages 34previous in the pipeline may be accumulated yielding a cumulativelikelihood score after the kth stage as L_k according to L_k=L_(k−1)+I_k.” '017 publication, paragraph [0034]. The likelihood score ateach stage L_k is rigidly based on the likelihood of the previous stageL_(k−1), in a pipeline fashion. Accumulation is presumed to always occurin the pipeline, and, as such, there is no need to determine whether thelikelihood scores ought to be accumulated. Therefore, there is noaccessing of any value to determine whether to accumulate.

BRIEF SUMMARY

Certain apparatuses, methods, systems, and instructions stored oncomputer-readable medium are presented for performing hardware-basedclassification. In at least one embodiment, a hardware engine isconfigurable to implement a plurality of cascade classifiers comprisingat least a first cascade classifier and a second cascade classifier, thehardware engine configurable to implement one or more stages within eachof the plurality of cascade classifiers. The first cascade classifierand the second cascade classifier may be configured to performclassification based on image data stored in a common image data buffer.The first cascade classifier may be configured to perform aclassification different from a classification performed by the secondcascade classifier. Also, the hardware engine may be capable ofconditionally branching from the first cascade classifier to the secondcascade classifier based on an outcome condition associated withevaluating of at least one stage of the first cascade classifier.

The image data stored in the common image data buffer may comprise imagedata representing a portion of an image, a portion of an integral image,or an integral image of a portion of the image. In one embodiment, thefirst cascade classifier is configured to perform object detection on afirst window size of the image data and the second cascade classifier isconfigured to perform object detection on a second window size of theimage data different from the first window size. In another embodiment,the first cascade classifier is configured to perform object detectionof a first object class and the second cascade classifier is configuredto perform object detection of a second object class different from thefirst object class. The outcome condition comprises a branch-on-failcondition corresponding to a failure by the at least one stage of thefirst cascade classifier to detect an object within a window of theimage data stored in the common image data buffer.

Each stage within the each of the plurality of cascade classifiers maycorrespond to a hardware stage storing stage data to enable the hardwareengine to implement each stage, wherein each hardware stage isassociated with a unique hardware stage identifier. Hardware stagescorresponding to similar logical stages of the first cascade classifierand the second cascade classifier may be assigned hardware stageidentifiers within a common range of values. A value may be recordedrepresenting a highest hardware stage identifier reached during anexecution of the hardware engine. The hardware engine may have access tomemory to enable storing of stage-related data usable by the hardwareengine across different cascades of the plurality of cascades. In oneembodiment, each stage within the first cascade classifier and thesecond cascade classifier employs a modular design having similarhardware capabilities as other stages within the first cascadeclassifier and second cascade classifier.

Also, in at least one embodiment, a hardware engine configurable toimplement multiple stages of a cascade classifier including a firststage and a second stage, wherein the hardware engine is configurable to(a) access a value indicative of whether to accumulate, and (b)responsive to the value indicative of whether to accumulate meeting acontinue evaluation condition, (i) access a first numeric value obtainedfrom evaluation of the first stage of the cascade classifier, (ii)accumulate the first numeric value with a second numeric value obtainedfrom evaluation of the second stage of the cascade classifier togenerate an accumulated value, and (iii) utilize the accumulated valueto determine an outcome for the second stage of the cascade classifier.

Upon determining an “early pass” condition based on the first numericvalue, the hardware engine may be configured to bypass evaluation of thesecond stage of the cascade classifier. The early pass condition maycomprise the first numeric value meeting an early pass threshold. In oneembodiment, the value indicative of whether to accumulate is the firstnumeric value obtained from the evaluation of the first stage of thecascade classifier. The continue evaluation condition may comprise thefirst numeric value failing to meet the early pass threshold.

Upon determining an “early fail” condition based on the first numericvalue, the hardware engine may be configured to bypass evaluation of thesecond stage of the cascade classifier. The early fail condition maycomprise the first numeric value failing to meet an early-fail-avoidthreshold. The continue evaluation condition may comprise the firstnumeric value meeting the early-fail-avoid threshold.

Furthermore, in at least one embodiment, a hardware engine isconfigurable to implement multiple stages of a cascade classifier. Here,the cascade classifier may support multiple paths of evaluation throughthe multiple stages of the cascade classifier, including a first path ofevaluation and a second path of evaluation. The hardware engine may beconfigurable to determine, based on a numeric value associated withevaluation of a stage along the first path, that the first path ofevaluation was taken.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the disclosure are illustrated by way of example. In theaccompanying figures, like reference numbers indicate similar elements.

FIG. 1 illustrates the operation of a traditional software cascadeclassifier;

FIG. 2 depicts multiple hardware engines configured to implementmultiple cascade classifiers, according to one embodiment of the presentdisclosure;

FIG. 3 depicts a single hardware engine configured to implement multiplecascade classifiers, according to another embodiment of the presentdisclosure;

FIG. 4 shows the use of cascade classifiers in a hierarchical objectmodel, according to one embodiment of the present disclosure;

FIG. 5 illustrates two cascade classifiers operating independently;

FIG. 6 illustrates a conditional branch operation from a first cascadeclassifier to a second cascade classifier, according to an embodiment ofthe present disclosure;

FIG. 7 illustrates the assignment of hardware stage identifiers within acommon range of values to similar logical stages of a first cascadeclassifier and a second cascade classifier;

FIG. 8 shows the selection of sub-windows based on edge alignment,according to one embodiment of the disclosure;

FIG. 9 shows the selection of sub-windows based on center alignment,according to another embodiment of the disclosure;

FIG. 10 illustrates an “early pass” operation performed over stages of acascade classifier, according to an embodiment of the presentdisclosure;

FIG. 11 illustrates an “early fail” operation performed over stages of acascade classifier, according to an embodiment of the presentdisclosure;

FIG. 12A is a low level hardware diagram of the registers and gatesinvolved in the update of a current stage register used in a traditionalcascade classifier;

FIG. 12B is a low level hardware diagram of the registers and gatesinvolved in the update of a current stage register used in a cascadeclassifier capable of a branch-on-fail operation, according oneembodiment of the disclosure;

FIG. 13A is a low level hardware diagram of the registers and gatesinvolved in one implementation of a basic accumulate operation;

FIG. 13B is a low level hardware diagram of the registers and gatesinvolved in an implementation of an accumulate-on-fail operation;

FIG. 14A is a flow chart showing a conditional branch sequence from afirst cascade classifier to a second cascade classifier, according to anembodiment of the present disclosure;

FIG. 14B is a more detailed flow chart showing a conditional branchsequence 1410 from a first cascade classifier to a second cascadeclassifier, according to an embodiment of the present disclosure.

FIG. 15 is a flow chart showing a process for performing classificationusing a value accumulated across stages, according to an embodiment ofthe disclosure;

FIG. 16 is a flow chart showing a process for resolving ambiguity in abranched cascade path while performing a classification task, accordingto an embodiment of the disclosure; and

FIG. 17 is a block diagram of internal components of a device that mayutilize the techniques and features described herein, according to anembodiment.

DETAILED DESCRIPTION

Several illustrative embodiments will now be described with respect tothe accompanying drawings, which form a part hereof. While particularembodiments, in which one or more aspects of the disclosure may beimplemented, are described below, other embodiments may be used andvarious modifications may be made without departing from the scope ofthe disclosure or the spirit of the appended claims.

FIG. 1 illustrates the operation of a traditional cascade classifier.Here, a “classifier” refers to a mechanism capable of generating adecision regarding whether the input data belongs one of a number ofpossible categories, or classifications. The input data may be, forexample, image data. The number of possible classifications may be two(binary classifier) or more (n-ary classifier). In various embodimentsof the present disclosure, the classification is performed using atrained machine learning (ML) classifier.

As shown, a cascade classifier 100 may have multiple stages, such as afirst stage 102, a second stage 104, and a third stage 106. Given aparticular sub-window of an image, the cascade classifier 100 attemptsto detect a particular object, such as a face, in the sub-window byperforming detection in cascaded stages. In each stage, a plurality offeatures that tend to indicate the presence of the target object arecalculated based on the sub-window of image data. Generally speaking,earlier stages tend to comprise “rougher” features that tend to yield ahigh detection rate for the target object. Just as an example, if thetarget object is a face, one feature that may be calculated in the firststage 102 may be a Haar-like feature that measures the difference in theintensity of pixel values between a region where the eyes are expectedto be (within the sub-window) and a region where the cheekbones of theface are expected to be (within the sub-window). This particular featureis chosen based on the fact that in most facial images, the eye regiongenerally contains relatively darker pixels when compared to thecheekbone region which generally contain relatively lighter pixels. Morethan one such feature may be calculated in the first stage 102. Themultiple features calculated in a stage produce values (e.g., pixelintensity differences) that may be combined, either by simple additionor by other operation(s), to yield a combined score for the stage. Thecombined score is then compared to a threshold. Based on the thresholdcomparison, a binary outcome is determined for the stage. The outcomemay be either that detection has been achieved for the stage (e.g., TRUEor “T”) or detection has not been achieved for the stage (e.g., FALSE or“F”). As the cascade classifier proceeds to later stages, more “refined”features may be used.

The cascade classifier 100 is referred to as being “cascaded” in thesense that each subsequent stage 104, 106, etc. is evaluated based onthe outcome of a previous stage. For instance, if the first stage 102results in detection (“T”), then the second stage 104 may be evaluated.In other words, if the “rough” features of the first stage 102 thatcorrespond to a face are sufficiently detected, as measured by thethreshold comparison within the first stage 102, then the cascadeclassifier 100 moves ahead to evaluate more refined features containedin the second stage 104. Otherwise, if the first stage 102 results in nodetection (“F”), then the sub-window is rejected. That is, if the“rough” features of the first stage 102 are not sufficiently detected,as measured by the threshold comparison within the first stage 102, thenthe cascade classifier concludes that the sub-window does not contain aface, without evaluating any subsequent stages such as the second stage104 or third stage 106.

The cascade classifier 100 is efficient in the sense that a sub-windowthat clearly does not contain a face can be rejected early, withoutevaluating all of the stages that exist in the classifier and thefeatures contained therein. A sub-window that does contain a face mayproceed through the cascade of stages, with confidence growing at eachstage that a face may be detected. Once a sufficient level of confidencehas been reached, e.g., by achieving detection for a certain number ofstages, the cascade classifier 100 may finally conclude that a face isdetected. While Haar-like features are used as an example in thediscussion above, other types of features may be used such as localbinary pattern (LBP), patch symmetric patterns, transitional LBPpatterns, center symmetric patterns, etc.

FIG. 2 depicts multiple hardware engines configured to implementmultiple cascade classifiers, according to one embodiment of the presentdisclosure. The hardware engines shown are examples of components thatmay constitute a hardware-based classification apparatus according tovarious embodiments of the present disclosure. As shown in the figure, afirst hardware engine 202 and second hardware engine 204 are configuredto implement a first cascade classifier (“Cascade 0”) and a secondcascade classifier (“Cascade 1”) by performing classification based onimage data stored in a common image data buffer 206. In one embodiment,the common image data buffer 206 may store a portion of an image 208that corresponds to an actual image captured by an image sensor, such asa camera. In another embodiment, the common image data buffer 206 maystore a portion of an integral image, or an integral image of a portionof an image, such as image 208. Here, an integral image refers to animage representing a summed-area table used to efficiently generate thesum of pixel values in a rectangular subset of a rectangular image. Forexample, the values of the pixel at (x,y) in the integral image may bedefined as:

${{I\left( {x,y} \right)} = {\sum_{\underset{y^{\prime} \leq y}{x^{\prime} \leq x}}{i\left( {x^{\prime},y^{\prime}} \right)}}},$

where i(x,y) is the value of the pixel at (x,y) in the original image.

The first hardware engine 202 and second hardware engine 204 mayimplement one or more stages within each of the two cascade classifiersin a variety of ways. In the embodiment shown in FIG. 2, the firsthardware engine 202 contains a set of registers, including a currentstage register 210 and memory register(s) 212 which keep track of datafor the first cascade classifier, Cascade 0. The operation of the firstcascade classifier and second cascade classifier is described in moredetail in subsequent sections, e.g., in the context of FIG. 5. Forexample, each stage in the first cascade classifier may be given anindex number, and the current stage register 210 may store a value thatrepresents the index number of the “current stage” that is presentlybeing evaluated in the first cascade classifier. Memory register(s) 212may store values such as accumulation values that may be used orgenerated by the execution of the first cascade classifier as itexecutes through its stages. Similarly, the second hardware engine 204contains a set of registers, including a current stage register 214 andmemory register(s) 216 which keep track of data for the second cascadeclassifier, Cascade 1.

In this embodiment, the first hardware engine 202 and second hardwareengine 204 may access a stage database 218 that stores data for eachstage of the plurality of stages associated with each cascadeclassifier. As shown in the figure, for stages associated with the firstcascade classifier, Cascade 0, the stage database 218 may include one ormore data registers or memory locations for storing data for stages 1through L-1. For stages associated with the second cascade classifier,Cascade 1, the stage database may include one or more data registers ormemory location for storing data for stages L through M. The stagedatabase 218 may store data for additional stages, such as data foradditional stages through stage N-1 and stage N, e.g., to accommodateadditional cascade classifiers. The data registers or memory locationsmay be used to store values such as flags, scores, thresholds, etc., tosupport the evaluation of the particular stage of the respective cascadeclassifier.

FIG. 3 depicts a single hardware engine configured to implement multiplecascade classifiers, according to another embodiment of the presentdisclosure. The hardware engine shown is an example of a component thatmay constitute a hardware-based classification apparatus according tovarious embodiments of the present disclosure. Here, a single hardwareengine 302 is configured to implement a first cascade classifier(“Cascade 0”) and a second cascade classifier (“Cascade 1”) byperforming classification based on image data stored in a common imagedata buffer 306. Similar to the arrangement shown in FIG. 2, the commonimage data buffer 306 may store a portion of an image 308 thatcorresponds to an actual image captured by a sensor such as a camera,just as an example. In another embodiment, the common image data buffer306 may store a portion of an integral image, or an integral image of aportion of an image, such as an image 308.

The single hardware engine 302 may implement one or more stages withineach of two cascade classifiers in a variety of ways. In the embodimentshown in FIG. 3, the single hardware engine 302 contains a set ofregisters, including a current stage register 310 and memory register(s)312 which keep track of data for the combined classifier that links twocascade classifiers, such as a first cascade classifier, Cascade 0, anda second cascade classifier, Cascade 1. The operation of such a combinedclassifier is described in more detail in subsequent sections, e.g., inthe context of FIG. 6. Here, each stage in the combined cascadeclassifier may be given an index number, and the current stage register310 may store a value that represents the index number of the “currentstage” that is presently being evaluated in the combined cascadeclassifier. Memory register(s) 312 may store values such as accumulationvalues that may be used or generated by the execution of the combinedcascade classifier as it executes through its stages. The stages of thecascade classifiers may be implemented in various ways. In oneembodiment, stages may be implemented as hardware, e.g., in the form ofspecific registers or portions of memory that may be associated withindex numbers. In other embodiments, stages may be implemented insoftware, e.g., as pointers or allocated memory locations which may belabeled using different names and/or indices.

According embodiments of the disclosure, a hardware engine may haveaccess to memory to enable storing of stage-related data usable by thehardware engine across different cascades. For example, in theembodiment shown in FIG. 3, the single hardware engine 302 may access astage database 318 that stores data for each stage of the plurality ofstages associated with the combined cascade classifier. As shown in thefigure, for stages associated with the first cascade classifier, Cascade0, the stage database 318 may include one or more data registers ormemory locations for storing data for stages 1 through L-1. For stagesassociated with the second cascade classifier, Cascade 1, the stagedatabase may include one or more data registers or memory location forstoring data for stages L through M. The stage database 318 may storedata for additional stages, such as data for additional stages throughstage N-1 and stage N, e.g., to accommodate additional cascadeclassifiers that may be linked in a similar manner, as part of thecombined cascade classifier. The data registers or memory locations maybe used to store values such as flags, scores, thresholds, etc., tosupport the evaluation of the particular stage of the respective cascadeclassifier.

FIG. 4 shows the use of cascade classifiers in a hierarchical objectmodel, according to one embodiment of the present disclosure. Eachcascade classifier may comprise one more stages. Here, the illustrativecascade classifier includes n stages, marked as H₁, H₂, . . . , H_(n).At each stage, based on the features calculated at that stage, adecision is made as to the detection or failure of detection of thetarget object. If detection is made (e.g., “Face”), the cascadeclassifier proceeds to the next stage. If no detection is made(e.g.,“Non-face), the cascade classifier exits its routine and concludesthat detection has failed, without proceeding to the next stage. Asshown in the figure, one or more cascade classifiers may be constructedin this manner and used together to build an object model. The one ormore cascade classifiers may be configured to detect the same objectaccording to different parameters or different object classes.

FIG. 5 illustrates in further detail two cascade classifiers operatingindependently. A first cascade classifier 502 and a second cascadeclassifier 504 are depicted. In one embodiment, the first cascadeclassifier 502 performs object detection (e.g., detect a face) on awindow of a first window size (e.g., window 0 in FIGS. 2 and 3), and thesecond cascade classifier 504 performs object detection (e.g., detect aface) on a window of a second window size (e.g., window 1 in FIGS. 2 and3) different from the first window size. In another embodiment, thefirst cascade classifier 502 performs object detection of a first objectclass (e.g., face versus body), while the second cascade classifier 504performs object detection of a second object class (e.g., humans versuscats) different from the first object class.

The first cascade classifier 502 is shown to include multiple stages,including stages 0, 1, 2, and 3. The second cascade classifier 504 isshown to include multiple stages as well, including stages 0, 1, 2, and3. The first cascade classifier 502 and the second cascade classifier504 may be implemented by use of one or more hardware engines, such ashardware engines 202 and 204 shown in FIG. 2 or single hardware engine302 shown in FIG. 3. Each of the first cascade classifiers 502 andsecond cascade classifier 504 operates independently, in a cascadedmanner. For example, in stage 0 of the first cascade classifier 502,certain features may be calculated and the resulting combined score forthe features may be compared to a threshold. If the threshold is met orsurpassed (as indicated by a solid arrow), indicating that detection atstage 0 is achieved, the first cascade classifier proceeds to stage 1.Otherwise, if the threshold is not met or surpassed (as indicated by thedashed arrow), indicating that detection at stage 0 is not achieved, thefirst cascade classifier 502 declares a “Fail” and exits withoutevaluating any subsequent stages. Subsequent stages, including stages 1,2, and 3, operation in like fashion. Depending on the result at eachstage, the first cascade classifier 502 either proceeds to a subsequentstage or declares a “Fail” and exists without evaluating any subsequentstages. The second cascade classifier 504 operations in a similarmanner. Each cascade classifier is efficient in the sense that it canreject a sub-window of image data (such as window 0 shown in FIG. 2 andFIG. 3) early, without evaluating all of the stages in subsequentstages, if a particular stage fails to meet the threshold set for thefeatures calculated for that stage.

According to conventional techniques, the hardware implementation ofindependent cascade classifiers, such as the first cascade classifier502 and second cascade classifier 504 shown in FIG. 5, typically followsa “hard-wired” approach that forces the evaluation path after eachstage, e.g., Stage 0, to either (1) proceed to the next stage, e.g.,Stage 1, or (2) exit the cascade classifier. Such an approach isillustrated in the operation of the cascade classifiers shown in FIG. 5.

FIG. 6 illustrates a conditional branch operation from a first cascadeclassifier to a second cascade classifier, according to an embodiment ofthe present disclosure. As before, a first cascade classifier 602 and asecond cascade classifier 604 are depicted. Similar to FIG. 5, the firstcascade classifier 602 and second cascade classifier 604 shown in FIG. 6may perform object detection on windows of different window sizes (e.g.,windows 0 and 1 in FIGS. 2 and 3). In another embodiment, the firstcascade classifier 602 performs object detection of a first object class(e.g., face versus body), while the second cascade classifier 604performs object detection of a second object class (e.g., humans versuscats) different from the first object class.

FIG. 6 illustrates that the first cascade classifier 602 and secondcascade classifier 604 may be implemented as a combined cascadeclassifier 606. In one embodiment, the combining of the two cascadeclassifiers is achieved through the use of a conditional branchoperation introduced into the capability of the hardware resourcesassociated with the cascade classifiers. Here, the conditional branchoperation is used to link the first cascade classifier 602 to the secondcascade classifier 604. For example, in stage 0 of the first cascadeclassifier 602, certain features may be calculated and the resultingcombined score for the features compared to a threshold. If thethreshold is met or surpassed (as indicated by a solid arrow),indicating that detection at stage 0 is achieved, the first cascadeclassifier proceeds to stage 1. This is similar to the operation shownin FIG. 5. However, if the threshold is not met or surpassed (asindicated by the dashed arrow), indicating that detection at stage 0 isnot achieved, the first cascade classifier 602 declares a “Fail” andbranches to stage 0 of the second cascade classifier 604. This contrastswith the logical flow shown in FIG. 5, where upon a “Fail” condition,the first cascade classifier simply exits. The other stages of the firstcascade classifier 602 operation in a similar manner. That is, upon a“Fail” condition (as indicated by the dashed arrow), the first cascadeclassifier 602 does not exit but instead branches to stage 0 of thesecond cascade classifier 604. Although all stages of the first cascadeclassifier 602 are illustrated as branching to the same stage of thesecond cascade classifier 604 (hardware stage 5; cascade 1, stage 0) forease of explanation, it is understood that, more generally, a fail atany given stage in first cascade classifier 602 could branch to anystage in the second cascade classifier 604. As such, the stage in secondcascade classifier 604 to which a given stage in first cascadeclassifier 602 branches on fail could, in some implementations, dependupon the given stage in the first cascade classifier 602.

Thus, FIG. 6 presents an example of performing classification with afirst cascade classifier based on image data stored in a common imagedata buffer and performing classification with a second cascadeclassifier based on the image data stored in the common image databuffer. Here, the classification performed by the second cascadeclassifier is different from the classification performed by the firstcascade classifier. According to an embodiment, performing theclassification with the first cascade classifier comprises (1)determining an outcome condition associated with evaluation of at leastone stage of one or more stages of the first cascade classifier and (2)responsive to the outcome condition matching a pass condition,proceeding to a next stage of the one or more stages of the firstcascade classifier. Responsive to the outcome condition matching a failcondition, the process may conditionally branch from the first cascadeclassifier to a stage of one or more stages of the second cascadeclassifier.

FIG. 12B, discussed in subsequent sections, provide further details ofan example of a hardware design to implement a conditional branchoperation such as that depicted in FIG. 6.

Operation at each stage of the second cascade classifier 604 is similarto that of FIG. 5. That is, if the threshold is met or surpassed (asindicated by a solid arrow), indicating that detection at the stage isachieved, the second cascade classifier 604 proceeds to the next stage.If the threshold is not met or surpassed (as indicated by the dashedarrow), indicating that detection at the stage is not achieved, thesecond cascade classifier 604 declares a “Fail” and exits.

As shown in FIG. 6, the resulting combined cascade classifier 606 hashardware stages associated with unique hardware stage identifiers 0through 9. Stages associated with hardware stage identifiers 0 through 3in the combined cascade classifier 606 correspond to logical stages 0through 3 of the first cascade classifier 602. Stages associated withhardware stage identifiers 5 through 8 in the combined cascadeclassifier 606 correspond to logical stages 0 through 3 of the secondcascade classifier 604.

A benefit of the conditional branch capability to branch from onecascade classifier to another is design flexibility. Different cascadeclassifiers of different lengths may be implemented, and the design maychange, without requiring a new hardware layout of the system. By makingstage(s) where conditional branch occurs configurable, as well as makingthe branch increment configurable, the overall design of the system isfreed from the inflexible nature of hard-wired cascade classifiers, inwhich stages and path between stages, once formed, are difficult toalter.

Furthermore, a modular design may be advantageously used to implementthe stages of the cascade classifier(s), such as those shown in FIG. 6.The hardware associated with each stage may be similarly structured andcontain similar capabilities. For example, each stage may be equipped tocalculate one or more features, generate a combined score from thecalculated features, compare the combined score to a threshold, performa conditional branch (e.g., branch on fail), etc. The operation of eachstage may be individually configurable, by supplying the stage with theparticular control parameters specific to that stage, such as featuredefinitions, threshold value, branch increment value, etc. The modularnature of the stages of the first and second cascade classifiers allowsthe overall system to be highly configurable. Such a design may serve toprovide additional flexibility, such that the same hardware can bere-configured to meet different or changing object detection needs.

The modular design of the stages may also allow different types ofstages to be implemented using the same design. For example, a “FakeStage” may be implemented using the same modular hardware resource of astandard stage. In the context of FIG. 6, such “Fake Stages” may beemployed to effectuate a graceful exit in the event that the targetobject is found by either the first cascade classifier 602 or the secondcascade classifier 604. More generally speaking, a “Fake Stage” may beused in different scenarios where the output is predetermined (e.g.,always pass or fail). In the embodiment depicted in the figure, a facemay be detected using either the first cascade classifier 602 (operatingon Window 0) or the second cascade classifier 604 (operating on Window1). Specifically, if stage 3 of the first cascade classifier 602 resultsin the threshold being met or surpassed (as indicated by a solid arrow),indicating that detection at stage 3 is achieved, the desired outcome isto reach the conclusion of “Found Object.” To implement such a logicalpath, a “Fake Stage” may be configured with an artificially highthreshold value, such that the result of the comparison operation isguaranteed to be “Fail” (as indicated by the dashed arrow). The branchincrement may be set so that the “Branch on Fail” operation leads to thecorresponding “Fake Stage” of the second cascade classifier 604. The“Fake Stage” of the second cascade classifier 604 may be configure withan artificially low threshold value, such that the threshold isguaranteed to be met or surpassed (as indicated by a solid arrow). Thus,detection of the target object by either the first cascade classifier602 or the second cascade classifier 604 would lead to the same “FoundObject” result. The exit upon detection of the target object is achievedwithout the need for any specialized stages. Instead, the “Fake Stages”and the real stages are all realized using the same modular stagedesign.

According to one embodiment, a capability to accumulate a value acrossstages may be introduced to the cascade classifier to address variousissues. One issue relates to the potential ambiguity of the pathtraveled upon reaching the “Found Object” result. For example, referringagain to FIG. 6, the “Found Object” result may be reached by traversinga path “X” through the first cascade classifier 602 (e.g., stagesassociated with hardware stage identifiers 0, 1, 2, 3, 4, and 9) or apath “Y” through the second cascade classifier 604 (e.g., stagesassociated with hardware stage identifiers 0, 5, 6, 7, 8, and 9). Suchan ambiguity may need to be resolved, for instance, if the first cascadeclassifier is configured to detect the target object in one sub-window(e.g., Window 0 in FIG. 2 or 3), and the second cascade classifier isconfigure to detect the target object in another sub-window (e.g.,Window 1 in FIG. 2 or 3). It may be necessary to ascertain not merelythe fact that the target object was detected, but which sub-window wasthe target object detected in. Thus, there may exist a need to resolvethe ambiguity between path “X” vs. path “Y.” To address this concern, acapability to accumulate a value across the combined cascade classifier706 may be introduced. FIGS. 13A and 13B, discussed in subsequentsections, provide further details of examples of hardware designs toimplement such a capability to accumulate a value across stages of thecombined cascade classifier.

The accumulate capability can be used to resolved the ambiguity oftraversed paths, by setting flag values in relevant stages that wouldindicate traversal through the particular stage. Just as an example, inFIG. 6, the ambiguity between path “X” through the first cascadeclassifier 602 (e.g., stages associated with hardware stage identifiers0, 1, 2, 3, 4, and 9) and path “Y” through the second cascade classifier604 (e.g., stages associated with hardware stage identifiers 0, 5, 6, 7,8, and 9) may be resolved by detecting whether a flag set by the stageassociated with hardware stage identifier 3 is present in theaccumulated value. The accumulated value for the combined cascadeclassifier 606 may be stored, e.g., as a 32-bit unsigned integer, in aparticular “accumulate” register (not shown in FIG. 6). The accumulatedvalue may be reset upon entering the combined cascade classifier 606. Ateach stage, the stage may add a number to the value stored in the“accumulate” register and write the resulting sum back into the“accumulate register.” In this simple example, all of the stages exceptthe stage associated with hardware stage identifier 3 would add “0” tothe accumulated value. By contrast, the stage associated with hardwarestage identifier 3 would add a nominal value, say “1000” to theaccumulated value. Upon reaching the “Found Object” result, theambiguity between path “X” and “Y” may be resolved by simply inspectingthe value in the “accumulate” register. An accumulate value of “1000” inthe register would indicate that the combined cascade classifier 606passed through the stage associated with hardware stage identifier 3,and the “X” path was traversed. An accumulated value of “0” wouldindicate that the combined cascade classifier 606 did not passed throughthe stage associated with hardware stage identifier 3, and the “Y” pathwas traversed. Here, the accumulated value is used as a “flag” in thegeneral sense that it serves as a status indicator. The particularvalues used can be different (e.g., value other than “1000”). Also theoperation does not necessarily have to be a pure addition operation.Just as an example, a modulo-n operation such as an modulo-2 sum may beused—e.g., 1 mod2 may indicate a path through the “X” path, and 0 mod2may indicate a path through the “Y” path.

FIG. 7 illustrates the assignment of hardware stage identifiers within acommon range of values to similar logical stages of a first cascadeclassifier and a second cascade classifier. As shown in the figure,various conditional branch operations may link a first cascadeclassifier 702 to a second cascade classifier 704, to form a combinedcascade classifier 706. Here, hardware stages of the combined cascadeclassifier 706 that correspond to similar logical stages of the firstcascade classifier 702 and the second cascade classifier 704 areassigned hardware stage identifiers within a common range of values.

For example, hardware stage identifiers 0, 1, 2, and 3 represent acommon range of “low” values, and they are assigned to logical stage 0of the first cascade classifier 702, a logical “Fake Stage” of the firstcascade classifier 702, logical stage 0 of the second cascade classifier704, and a “Fake Stage” of the second cascade classifier 704,respectively. Hardware stage identifiers 4, 5, 6, and 7 represent acommon range of “medium” values, and they are assigned to logical stage1 of the first cascade classifier 702, a logical “Fake Stage” of thefirst cascade classifier 702, logical stage 1 of the second cascadeclassifier 704, and a “Fake Stage” of the second cascade classifier 704,respectively. Finally, hardware stage identifiers 8, 9, 10, 11, 12, and13 represent a common range of “high” values, and they are assigned tological stage 2 of the first cascade classifier 702, logical stage 3 ofthe first cascade classifier 702, a logical “Fake Stage” of the firstcascade classifier 702, logical stage 2 of the second cascade classifier704, logical stage 3 of the second cascade classifier 704, and a “FakeStage” of the second cascade classifier 704, respectively. Hence,hardware stages implementing logical stages of the first cascadeclassifier and similar logical stages of the second cascade classifierare assigned hardware stage identifiers within a common range of values.As such, both first cascade classifier 702 and second cascade classifier704 have similar logical stages, both have a stage 0. The hardwarestages implementing these logical stages are both in a “low” range ofvalues, namely 0 and 2.

By assigning hardware stages corresponding to similar logical stages ofdifferent cascade classifiers to hardware stage identifiers within acommon range of values, the hardware stage identifier becomes a usefulindicator of the depth to which object detection reached within thelogical cascade classifiers. By recording the highest hardware stageidentifier reached over the course of an object detection, an indicatorof the strength of the detection may be obtained. If object detectiondid not reach very deep within the logical cascade classifiers, asindicated the highest recorded hardware identifier being a low value, itmay be concluded that there was weak detection of the target object. Onthe other hand, if object detection reached deep within the logicalcascade of classifiers, as indicated by the highest recorded hardwareidentifier being a high value, it may be concluded that there was astrong detection of the target object. The highest hardware stageidentifier recorded, for example, may be the highest stage identifierrecorded during an execution of the hardware engine, e.g., during anexecution of the first cascade classifier and the second cascadeclassifier. It is during such an execution that an object may bedetected. Hence, the hardware engine can be configured to record a valuerepresenting a highest hardware stage identifier reached during anexecution of the hardware engine.

While the first cascade classifier and the second cascade classifier areshown here and in previous figures as having an equal number of stages,in other embodiments, the number of stages present in different cascadeclassifiers may be different. Also, while examples depicting two cascadeclassifiers are described for ease of illustration, the presentdisclosure encompasses embodiments that implement more than two cascadeclassifiers.

FIG. 8 shows the selection of sub-windows based on edge alignment,according to one embodiment of the disclosure. Typically, an objectdetector such as each of the cascade classifiers disclosed herein isused to scan across an image (e.g., image 208 in FIG. 2) at multiplescales and locations. One way to scale the detector is by implementingdifferently sized cascade classifiers that operate on sub-windows ofdifferent sizes. For example, FIG. 8 shows three different sizes ofsub-windows: 20 pixels by 20 pixels (20×20), 25 pixels by 25 pixels(25×25), and 32 pixels by 32 pixels (32×32). In this embodiment, eachsuccessively smaller window is determined by selecting a sub-portion ofthe present window based on edge alignment—i.e., alignment along twoadjacent edges. As mentioned previously, the image data contained ineach sub-window used may be captured image data, a portion of anintegral image, or an integral image of a portion of image.

FIG. 9 shows the selection of sub-windows based on center alignment,according to another embodiment of the disclosure. Once again, thefigure shows three different sizes of sub-windows: 20 pixels by 20pixels (20×20), 25 pixels by 25 pixels (25×25), and 32 pixels by 32pixels (32×32). In this embodiment, each successively smaller window isdetermined by selecting a sub-portion of the present window based oncenter alignment. That is, each successively smaller sub-window iscentered on the previous, larger sub-window.

FIG. 10 illustrates an “early pass” operation performed over stages of acascade classifier, according to an embodiment of the presentdisclosure. As shown in FIG. 10, a cascade classifier 1000 comprises anumber of stages. Here, cascade classifier 1000 is described asimplementing a logical cascade classifier (“Cascade 0”) which containslogical stages 0, 1 a, 1 b, 2, and 3. Cascade classifier 1000 may bebuilt using hardware stages, labeled with hardware stage identifiers 0,1, 2, 3, 4, and 5. For ease of explanation, the discussion of FIG. 10below refers to logical stages 0, 1 a, 1 b, 2, and 3. It should beunderstood that these stages correspond to hardware stages 0, 1, 3, 4,and 5, respectively.

Generally speaking, the “early pass” operation allows for one or morefeatures to be bypassed if their evaluation is deemed unnecessary, inorder to improve efficiency. For example, take the case of a logicalstage (“stage 1”—not shown in FIG. 10) that contains 10 features. If thefirst three features are particularly important, and 90% of images (orsub-windows) would pass stage 1 if they passed these three features, thethree features may be placed in a separate stage, i.e., logical stage 1a. The rest of the seven features may be placed in another stage, i.e.,logical stage 1 b. If a particular sub-window passes an early passthreshold set with respect to the three features, i.e., passes logicalstage 1 a, calculating the other seven features may be avoided. That is,logical stage 1 b may be bypassed. Doing so would improve the efficiencyof the cascade classifier. This is depicted in FIG. 10. At logical stage1 a, if the early pass threshold is met (as indicated by a solid arrow,indicating an early pass condition), indicating that detection at thisstage is achieved, logical stage 1 b is bypassed (hence, evaluation oflogical stage 1 b is bypassed), and execution proceeds to logical stage2. As shown in the figure, the bypass routine may be implemented using a“Fake Stage” (hardware stage index “2”), which may be configured with athreshold that guarantees a “branch on fail” outcome (as indicated by adashed arrow), which leads to logical stage 2.

On the other hand, at logical stage 1 a, if the sub-window does not passthe threshold set with respect to the three features, i.e., fails topass the stage, it may be necessary to calculate the other sevenfeatures. Hence, if the early pass threshold at logical stage 1 a is notmet (continue evaluation condition), the stage fails (as indicated by adashed arrow), and logical stage 1 b is evaluated, i.e., evaluation ofstage 1 continues by evaluating stage 1 b. At this particular juncture,evaluation of logical stage 1 b may involve all ten features. In atraditional cascade classifier design, all ten features would becalculated, including the three features that had previously beencalculated in logical stage 1 a. According to an embodiment of thepresent disclosure, a capability to accumulate a numeric value acrossstages may be employed to avoid such re-calculation.

A capability to accumulate a value across stages of a cascade classifiermay involve, for example, a hardware engine configurable to(a) access avalue indicative of whether to accumulate, (b) responsive to the valueindicative of whether to accumulate meeting a continue evaluationcondition, (i) access a first numeric value obtained from evaluation ofthe first stage, e.g., logical stage 1 a, of the cascade classifier,(ii) accumulate the first numeric value with a second numeric valueobtained from evaluation of the second stage, e.g., logical stage 1 b,of the cascade classifier to generate an accumulated value, and (iii)utilize the accumulated value to determine an outcome for the secondstage of the cascade classifier. In this example, the value indicativeof whether to accumulate is the first numeric value obtained fromevaluation of three features in the first stage of the cascadeclassifier, e.g., logical stage 1 a.

Referring back to FIG. 10, logical stage 1 a may utilize the accumulatecapability to pass the value or score obtained from calculating itsthree features to logical stage 1 b. Logical stage 1 b may then use theaccumulate feature to add the score from logical stage 1 a to the scoregenerated from calculating the remaining seven features. The result isan accumulated score that reflects calculation of all ten featurespresent in logical stages 1 a and 1 b. This can be achieved withoutperforming re-calculation of the first three features. The accumulatedscore may then be evaluated against a threshold at logical stage 1 b. Asshown in FIG. 10, if the threshold at logical stage 1 b is not met (asindicated by a dashed arrow), the cascade classifier exits with the“fail” outcome. If the threshold at logical stage 1 b is met (asindicated by a solid arrow), the cascade classifier proceeds to logicalstage 2, the same place where logical stage 1 a was directed uponmeeting the “early pass” condition for bypassing logical stage 1 b.

FIG. 11 illustrates an “early fail” operation performed over stages of acascade classifier, according to an embodiment of the presentdisclosure. As shown in FIG. 11, a cascade classifier 1100 comprises anumber of stages. Here, cascade classifier 1100 is described asimplementing a logical cascade classifier (“Cascade 0”) which containslogical stages 0, 1 a, 1 b, 2, and 3. Cascade classifier 1100 may bebuilt using hardware stages, labeled with hardware stage identifiers 0,1, 2, 3, and 4. It should be understood that logical stages 0, 1 a, 1 b,2, and 3 correspond to hardware stages 0, 1, 3, and 4, respectively.

Generally speaking, the “early fail” operation allows for early exit ofthe cascade classifier. For example, take the case of a logical stage(“stage 1”—not shown in FIG. 11) that contains 10 features. If notpassing any of the first three features means there is no chance thatthe stage 1 can be passed, then the three features may be placed into aseparate stage, i.e., logical stage 1 a. The rest of the seven featuresmay be placed in another stage, i.e., logical stage 1 b. Logical stage 1a may test whether any of the first three features passed. For example,each of the first three features may be assigned a score of 100. Each ofthe remaining seven features may be assigned a score of 20. Thethreshold (for example an early-fail-avoid threshold) for logical stage1 a may be set at 100. Thus, if none of the first three featurespasses—i.e., logical stage 1 a fails to meets its early-fail-avoidthreshold of 100 (as indicated by a dashed arrow), the cascadeclassifier performs the “early fail” operation to reach afailure-to-detect outcome and exists. On the other hand, if one or moreof the first three features passes (a continue evaluationcondition)—i.e., the logical stage 1 a meets its early-fail-avoidthreshold of 100 (as indicated by the solid arrow), the cascadeclassifier moves to logical stage 1 b (evaluation of stage 1 continuesat stage 1 b). At this juncture, evaluation of logical stage 1 b mayinvolve all ten features. In a traditional cascade classifier, all tenfeatures would be calculated, including the three features that hadpreviously been calculated in logical stage 1 a. According to anembodiment of the present disclosure, a capability to accumulate anumeric value across stages may be employed to avoid suchre-calculation.

As mentioned earlier, a capability to accumulate a value across stagesof a cascade classifier may involve, for example, a hardware engineconfigurable to (a) access a first numeric value obtained fromevaluation of the first stage of the cascade classifier, (b) accumulatethe first numeric value with a second numeric value obtained fromevaluation of the second stage of the cascade classifier to generate anaccumulated value, and (c) utilize the accumulated value to determine anoutcome for the second stage of the cascade classifier.

Referring back to FIG. 11, logical stage 1 a may utilize the accumulatecapability to pass the value or score obtained from calculating thethree features to logical stage 1 b. Logical stage 1 b may then use theaccumulate capability to add the score from logical stage 1 a to thescore generated from calculating the remaining seven features. Theresult is an accumulated score that reflects calculation of all tenfeatures present in logical stages 1 a and 1 b, without re-calculatingthe first three features (thereby saving time, power, etc.). Theaccumulated score may then be evaluated against a threshold at logicalstage 1 b. As shown in FIG. 11, if the threshold at logical stage 1 b isnot met (as indicated by a dashed arrow), the cascade classifier exitswith the “Fail” outcome. If the threshold at logical stage 1 b is met(as indicated by a solid arrow), the cascade classifier proceeds tological stage 2.

FIG. 12A is a low level hardware diagram of the registers and gatesinvolved in the update of a current stage register used in a traditionalcascade classifier. The components shown in FIGS. 12A and/or 12B may bepart of a hardware-based classification apparatus according to variousembodiments of the present disclosure. The design shown in FIG. 12Aincludes a register 1202 for storing an integer identifying the stagenumber of the current stage being evaluated in the cascade classifier,as well as an adder 1204 for adding a increment value of “1” to thecurrent stage number. The output of the adder 1204 is fed back to theregister 1202, so that the result of the addition operation can bewritten back to register 1202. This design may be appropriate for atraditional cascade classifier that always increments the stage numberby “1” and may not be suitable for a cascade classifier capable ofbranching or other operations that require the current stage number tobe incremented by a value other than “1.” The “Clear on Fail” signalserves as a reset for clearing the current stage number to a defaultvalue, e.g., “0.” The reset may be employed, for example, upon theinitiation of a cascade classifier.

FIG. 12B is a low level hardware diagram of the registers and gatesinvolved in the update of a current stage register used in a cascadeclassifier capable of a branch-on-fail operation, according to oneembodiment of the disclosure. As shown, the design includes a register1212 for storing an integer identifying the current stage number. Anexample of register 1212 may be the current stage register 310 found inthe hardware engine 302 shown in FIG. 3. The design further includes anAND gate 1214, a multiplexer 1216, and an adder 1218. A “Clear on Fail”signal and a Not(“Branch on Fail”) signal are provided as inputs to theAND gate 1214. As discussed, the “Clear on Fail” signal may serve as areset for clearing the current stage number to a default value, e.g.,“0.” Here, the current stage number may only be allowed to be reset ifno branch on fail operation is in progress. An increment value of “1”and a value of “deltaStage” are provided as alternative inputs to themultiplexer 1216. A signal represented by “Branch on Fail” AND “FAIL” isprovided as the selector input to the multiplexer 1216. Thus, if thecurrent stage is a “Branch on Fail” branch, and the output condition ofthe stage is “FAIL,” then the appropriate branch increment value, asrepresented by “deltaStage,” is selected by the multiplexer 1216.Otherwise, the value of “1” is selected by the multiplexer 1216. Theoutput of the multiplexer 1216 is provided as one of the inputs to theadder 1218. The other input of the adder 1218 comes from the output ofthe current stage register 1212. Adder 1218 adds either the value “1” orthe “deltaStage” increment value to the current stage number found inregister 1212 and writes the result of the addition operation back toregister 1212. Thus, the current stage number is updated to accommodatea branch on fail operation, if one is in effect.

FIG. 13A is a low level hardware diagram of the registers and gatesinvolved in one implementation of a basic accumulate operation. Thecomponents shown in FIGS. 13A and/or 13B may be part of a hardware-basedclassification apparatus according to various embodiments of the presentdisclosure. As shown in FIG. 13A, the design includes a register 1302and an adder 1304. A “Clear on Next Stage” signal may serve as a resetfor clearing the accumulated score to a default value, e.g., “0” whenneeded. The adder 1304 adds the “Feature Score” obtained from evaluationof the current stage to the accumulated score found in the register1302, and the result of the addition is stored back to the register1302. In this manner, the score stored in the register 1302 may beaccumulated over multiple stages.

FIG. 13B is a low-level hardware diagram of the registers and gatesinvolved in an implementation of an accumulate-on-fail operation. Asshown, the design includes a register 1312, an AND gate 1314, and anadder 1316. The “Clear on Next” signal is logically AND'ed with a signalrepresenting NOT (“Accumulate on Fail”), and the result is provided as areset signal for clearing the accumulated score to a default value,e.g., “0” when needed. Thus, when “Accumulate on Fail” is active, theregister 1312 is kept from being reset, and the score is accumulated.Similar as above, the adder 1316 adds the “Feature Score” obtained fromevaluation of the current stage to the accumulated score found in theregister 1312, and the result of the addition is stored back to theregister 1312. In this manner, the score stored in the register 1312 maybe accumulated over multiple stages.

FIG. 14A is a flow chart showing a conditional branch sequence 1400 froma first cascade classifier to a second cascade classifier, according toan embodiment of the present disclosure. As shown in FIG. 14A, thesequence 1400 comprises illustrative steps 1402, 1404, 1406, and 1408.At step 1402, the system determines an outcome condition associated withexecution or evaluation of at least one stage of one or more stages of afirst cascade classifier. At a step 1404, the system evaluates whetherthe outcome condition matches a pass condition or a fail condition.Means for performing steps 1402 and 1404 can, but not necessarily,include, for example, hardware engine 202 in FIG. 2, hardware engine 302in FIG. 3, processing unit(s) 1710 in FIG. 17, and/or digital signalprocessing unit (DSP) 1720 in FIG. 17. In step 1406, responsive to theoutcome condition matching a pass condition, the system proceeds to anext stage of the one or more stages of the first cascade classifier. Instep 1408, responsive to the outcome condition matching a failcondition, the system conditionally branches from the first cascadeclassifier to a stage of one or more stages of a second cascadeclassifier. Means for performing steps 1406 and 1408 can, but notnecessarily, include, for example, lower level hardware components suchas the circuitry shown in FIG. 12B, hardware engine 202 in FIG. 2,hardware engine 302 in FIG. 3, processing unit(s) 1710 in FIG. 17,and/or digital signal processing unit (DSP) 1720 in FIG. 17. Accordingto an embodiment, the first cascade classifier and the second cascadeclassifier may be configured to perform classification based on imagedata stored in a common image data buffer. Also, the first cascadeclassifier may be configured to perform classification different from aclassification performed by the second classifier.

Examples of sequence 1400 include those illustrated in FIG. 6, with thefirst cascade classifier being classifier 602, and the second cascadeclassifier being classifier 604. For instance, in stage 0 of the firstcascade classifier 602, certain features may be calculated and theresult evaluated against a threshold. The outcome condition may match apass condition if the threshold is met or surpassed, indicatingdetection of a target feature, and the system proceeds to the next stage(stage 1) of the first cascade classifier 602. On the other hand, theoutcome condition may match a fail condition if the threshold is not metor surpassed, indicating failure to detect the target feature, and thesystem conditionally branches from the first cascade classifier 602 to astage (stage 0) of the second cascade classifier 604. Further detailsregarding the operation of the first cascade classifier 602 and secondcascade classifier 604 are discussed in previous sections with respectto FIG. 6.

FIG. 14B is a more detailed flow chart showing a conditional branchsequence 1410 from a first cascade classifier to a second cascadeclassifier, according to an embodiment of the present disclosure.Conditional branch sequence 1410 shown in FIG. 14B may be an example ofthe conditional branch sequence 1400 shown in FIG. 14A. As such, detailsmentioned previously with respect to the conditional sequence 1400 mayalso be applicable to conditional sequence 1410. As shown, conditionalsequence 1410 comprises illustrative steps 1412 and 1414. At step 1412,the system performs classification with a first cascade classifier basedon image data stored in a common image data buffer. Step 1412 maycomprise sub-steps 1416, 1418, and 1420. At sub-step 1416, the systemdetermines an outcome condition associated with evaluation of at leastone stage of one or more stages of the first cascade classifier. Atsub-step 1418, the system determines whether the outcome conditionmatches a pass condition or a fail condition. At sub-step 1420,responsive to the outcome condition matching a pass condition, thesystem proceeds to and performs operations at a next stage of the one ormore stages of the first cascade classifier. Means for performing step1412, including sub-steps 1416, 1418, and 1420 can, but not necessarily,include, for example, hardware engine 202 in FIG. 2, hardware engine 302in FIG. 3, processing unit(s) 1710 in FIG. 17, and/or digital signalprocessing unit (DSP) 1720 in FIG. 17.

Responsive to the outcome condition matching a fail condition, thesystem conditionally branches from the first cascade classifier to astage of one or more stages of the second cascade classifier. Here, atstep 1414, the system performs classification with a second cascadeclassifier based on the image data stored in the common image databuffer. Step 1414 may comprise sub-step 1422. Thus, responsive to theoutcome condition matching the fail condition, the system mayconditionally branch to sub-step 1422. At sub-step 1422, the systemperforms operations at a stage of one or more stages of the secondcascade classifier. Means for performing step 1414, including sub-step1422 can, but not necessarily, include, for example, lower levelhardware components such as the circuitry shown in FIG. 12B, hardwareengine 202 in FIG. 2, hardware engine 302 in FIG. 3, processing unit(s)1710 in FIG. 17, and/or digital signal processing unit (DSP) 1720 inFIG. 17. According to an embodiment, the first cascade classifier andthe second cascade classifier may be configured to performclassification based on image data stored in a common image data buffer.Also, the first cascade classifier may be configured to performclassification different from a classification performed by the secondclassifier.

According to an embodiment of the disclosure, at steps 1412 and 1414,performing the classification with the first cascade classifiercomprises performing object detection on a first window size of theimage data, and performing the classification with the second cascadeclassifier comprises performing object detection on a second window sizeof the image data different from the first window size. Means forperforming object detection on the first window size of the image andperforming object detection on a second window size of the image datadifferent from the first window size can, but not necessarily, include,for example, hardware engines 202 and 204 in FIG. 2, hardware engine 302in FIG. 3, processing unit(s) 1710 in FIG. 17, and/or digital signalprocessing unit (DSP) 1720 in FIG. 17.

According to an embodiment of the disclosure, at steps 1412 and 1414,performing the classification with the first cascade classifiercomprises performing object detection of a first object class, andperforming the classification with the second cascade classifiercomprises performing object detection of a second object class differentfrom the first object class. Means for performing object detection ofthe first object class and performing object detection of the secondobject class can, but not necessarily, include, for example, hardwareengines 202 and 204 in FIG. 2, hardware engine 302 in FIG. 3, processingunit(s) 1710 in FIG. 17, and/or digital signal processing unit (DSP)1720 in FIG. 17.

FIG. 15 is a flow chart showing a process 1500 for performingclassification using a value accumulated across stages, according to anembodiment of the disclosure. As shown, process 1500 involves steps1502, 1504, and 1506. Step 1506 further comprises sub-steps 1508, 1510,and 1512. At step 1502, a first stage of a cascade classifier isevaluated to obtain a first numeric value. At step 1504, a valueindicative of whether to accumulate is accessed. At step 1506,responsive to the value indicative of whether to accumulate meeting acontinue evaluation condition, sub-steps 1508, 1510, and 1512 areperformed. Means for performing steps 1502,1504, and 1506 can, but notnecessarily, include, for example, lower level hardware components suchas the circuitry shown in FIG. 13, hardware engine 202 in FIG. 2,hardware engine 302 in FIG. 3, processing unit(s) 1710 in FIG. 17,and/or digital signal processing unit (DSP) 1720 in FIG. 17. At sub-step1508, the process involves accessing the first numeric value obtainedfrom evaluating the first stage of the cascade classifier. At sub-step1510, the process involves accumulating the first numeric value with asecond numeric value obtained during evaluating a second stage of thecascade classifier, to generate an accumulated value. At sub-step 1512,the process involves utilizing the accumulated value to determine anoutcome for the second stage of the cascade classifier. Means forperforming sub-steps 1508, 1510, and 1512 can, but not necessarily,include, for example, lower level hardware components such as thecircuitry shown in FIG. 13B, hardware engine 202 in FIG. 2, hardwareengine 302 in FIG. 3, processing unit(s) 1710 in FIG. 17, and/or digitalsignal processing unit (DSP) 1720 in FIG. 17.

One example of the process 1500 is the “early pass” operation shown inFIG. 10, which is described in detail previously. In this example, thefirst stage of the cascade classifier may be stage 1 a shown in FIG. 10.Evaluation of the features associated with stage la may yield a firstnumeric value. Then, responsive to a condition, e.g., the first numericvalue meeting or not meeting an “early pass” threshold, a second stageof the cascade classifier is evaluated. Here, the second stage may bestage 1 b shown in FIG. 10. In one pass of the cascade classifier, thefirst numeric value may not meet the “early pass” threshold. In such acase, “early pass” is not possible, and stage 1 b is not bypassed.Instead, stage 1 b is evaluated. Here, evaluating stage 1 b may comprisea number of sub-steps, including accessing the first numeric valueobtained from evaluating stage 1 a, accumulating the first numeric valuewith a second numeric value obtained during evaluation of the secondstage, i.e., stage 1 b of FIG. 10, to generate an accumulated value, andusing the accumulated value to determine the outcome of stage 1 b. Inanother pass of the cascade classifier, the first numeric value may meetthe “early pass” threshold. In such a case, “early pass” is possible,and stage 1 b is bypassed. When the cascade classifier is executedmultiple times, operation over the multiple executions may involve atleast one execution in which “early pass” occurs and at least oneexecution in which “early pass” does not occur. Thus, an overall processmay be viewed as including both “early pass” steps and steps associatedwith not performing “early pass.”

Another example of the process 1500 is the “early fail” operation shownin FIG. 11, which is also described in detail previously. In thisexample, the first stage of the cascade classifier may be stage 1 ashown in FIG. 11. Evaluation of the features associated with stage lamay yield a first numeric value. Then, responsive to a condition, e.g.,the first numeric value meeting or not meeting an “early fail”threshold, a second stage of the cascade classifier is evaluated. Here,the second stage may be stage 1 b shown in FIG. 11. In one pass of thecascade classifier, the first numeric value may not meet the “earlyfail” threshold. In such a case, “early fail” is not possible, and stage1 b is evaluated. Here, evaluating stage 1 b may comprise a number ofsub-steps, including accessing the first numeric value obtained fromevaluating stage 1 a, accumulating the first numeric value with a secondnumeric value obtained during evaluation of the second stage, i.e.,stage 1 b of FIG. 11, to generate an accumulated value, and using theaccumulated value to determine the outcome of stage 1 b. In another passof the cascade classifier, the first numeric value may meet the “earlyfail” threshold. In such a case, “early fail” is possible, and thecascade classifier exits execution with the “failed” outcome. When thecascade classifier is executed multiple times, operation over themultiple executions may involve at least one execution in which “earlyfail” occurs and at least one execution in which “early fail” does notoccur. Thus, an overall process may be viewed as including both “earlyfail” steps and steps associated with not performing “early fail.”

FIG. 16 is a flow chart showing a process 1600 for resolving ambiguityin a branched cascade path while performing a classification task,according to an embodiment of the disclosure. As shown, process 1600involves steps 1602, 1604, and 1606. At step 1602, the process involvesevaluating a plurality of stages of a cascade classifier having multiplestages. An example of such a cascade classifier is a combined cascadeclassifier 606 comprising cascade classifiers 602 and 604 shown in FIG.6. In step 1604, multiple paths of evaluation through the multiplestages of the cascade classifier are supported, including a first pathof evaluation and a second path of evaluation. Referring again to FIG.6, an example of a first path of evaluation may be a path “X” throughstages associated with hardware stage identifiers 0, 1, 2, 3, 4, and 9.An example of a second path of evaluation maybe path “Y” through stagesassociated with hardware stage identifiers 0, 5, 6, 7, 8, and 9. In step1606, the process involves determining, based on a numeric valueassociated with evaluation of a stage along the first path, that thefirst path of evaluation was taken. In the same example, the accumulatedvalue for the combined cascade classifier 606 may be stored, e.g., as a32-bit unsigned integer. The accumulated value may be reset uponentering the combined cascade classifier 606. At each stage, the stagemay add a number to the value stored in the “accumulate” register andwrite the resulting sum back into the “accumulate register.” In thissimple example, all of the stages except the stage associated withhardware stage identifier 3 would add “0” to the accumulated value. Bycontrast, the stage associated with hardware stage identifier 3 wouldadd a nominal value, say “1000” to the accumulated value. Upon reachingthe “Found Object” result, the ambiguity between path “X” and “Y” may beresolved by simply inspecting the value in the accumulate register. Anaccumulate value of “1000” in the register would indicate that thecombined cascade classifier 606 passed through the stage associated withhardware stage identifier 3, and the “X” path was traversed. Means forperforming steps 1602, 1604, and 1606 can, but not necessarily, include,for example, hardware engine 202 in FIG. 2, hardware engine 302 in FIG.3, processing unit(s) 1710 in FIG. 17, and/or digital signal processingunit (DSP) 1720 in FIG. 17.

The steps shown in FIGS. 14A, 14B, 15, and 16, can be implemented inhardware using digital logic, state machine(s), programmable logicdevice(s) such as field programmable gate array (FPGA), and/or specialpurpose processor(s) or application-specific integrated circuit(s) aswell as in software where, for example, general purpose processor(s)execute programmed instructions, or any combination thereof, includingillustrative components and devices specifically described in thepresent disclosure.

The methods or processes for classification described in the presentdisclosure, including those described in connection with FIGS. 14A, 14B,15, and 16, may be performed using hardware such as hardware engines 202and 204 shown in FIG. 2 and hardware engine 302 shown in FIG. 3 andassociated cascades classifiers. Use of hardware to perform suchtechniques may be associated with greater speed, efficiency, and lowerpower consumption. However, the methods and processes for classificationdescribed in the present disclosure may also be performed using one ormore processors, such as general purpose processing units, carrying outprogrammed instructions, according to other embodiments of thedisclosure.

FIG. 17 is a block diagram of internal components of a device 1700 thatmay utilize the techniques and features described herein, according toan embodiment. It should be noted that FIG. 17 is meant only to providea generalized illustration of various components, any or all of whichmay be utilized as appropriate. It can be noted that, in some instances,components illustrated by FIG. 17 can be localized to a single physicaldevice and/or distributed among various networked devices, which may bedisposed at different physical locations.

The device 1700 is shown comprising hardware elements that can beelectrically coupled via a bus 1705 (or may otherwise be incommunication, as appropriate). The hardware elements may include one ormore processing unit(s) 1710 which can include without limitation one ormore general-purpose processors, one or more special-purpose processors(such as digital signal processing (DSP) chips, graphics accelerationprocessors, application specific integrated circuits (ASICs), and/or thelike), and/or other processing structure or means. In the embodimentsdescribed above, the one or more processing unit(s) 1710 may incorporatehardware engines such as hardware engines 202 and 204 shown in FIG. 2 orhardware engine 302 shown in FIG. 3. As shown in FIG. 17, someembodiments may have a separate DSP 1720, depending on desiredfunctionality. Alternatively or additionally, DSP 1720 may incorporatehardware engines such as hardware engine 202 and 204 shown in FIG. 2 orhardware engine 302 shown in FIG. 3. The device 1700 also can includeone or more input devices 1770, which can include without limitation atouch screen, a touch pad, microphone, button(s), dial(s), switch(es),and/or the like; and one or more output devices 1715, which can includewithout limitation a display, light emitting diode (LED), speakers,and/or the like.

The device 1700 might also include a wireless communication interface1730, which can include without limitation a modem, a network card, aninfrared communication device, a wireless communication device, and/or achipset (such as a Bluetooth™ device, an 802.11 device (i.e., a deviceoperating in accordance with the Institute of Electrical and ElectronicsEngineers (IEEE) 802.11 specification), an 802.16.4 device, a WiFidevice, a WiMax device, cellular communication facilities, etc.), and/orthe like. The wireless communication interface 1730 may permit data tobe exchanged with a network, wireless access points, other computersystems, and/or any other electronic devices described herein. Thecommunication can be carried out via one or more wireless communicationantenna(s) 1732 that send and/or receive wireless signals 1734.

Depending on desired functionality, the wireless communication interface1730 can include separate transceivers to communicate with basetransceiver stations (e.g., base stations of a cellular network) and/oraccess point(s). These different data networks can include variousnetwork types. Additionally, a Wireless Wide Area Network (WWAN) may bea Code Division Multiple Access (CDMA) network, a Time Division MultipleAccess (TDMA) network, a Frequency Division Multiple Access (FDMA)network, an Orthogonal Frequency Division Multiple Access (OFDMA)network, a Single-Carrier Frequency division Multiple Access (SC-FDMA)network, a WiMax (IEEE 802.16), and so on. A CDMA network may implementone or more radio access technologies (RATs) such as cdma2000,Wideband-CDMA (W-CDMA), and so on. Cdma2000 includes IS-95, IS-2000,and/or IS-856 standards. A TDMA network may implement Global System forMobile Communications (GSM), Digital Advanced Mobile Phone System(D-AMPS), or some other radio access technology (RAT). An OFDMA networkmay employ Long Term Evolution (LTE), LTE Advanced, and so on. LTE, LTEAdvanced, GSM, and W-CDMA are described in documents from 3GPP. Cdma2000is described in documents from a consortium named “3rd GenerationPartnership Project 2” (3GPP2). 3GPP and 3GPP2 documents are publiclyavailable. A Wireless Local Area Network (WLAN) may also be an IEEE802.11x network, and a Wireless Personal Area Network (WPAN) may be aBluetooth network, an IEEE 802.16x, or some other type of network. Thetechniques described herein may also be used for any combination ofWWAN, WLAN and/or WPAN.

The device 1700 can further include sensor(s) 1740. Such sensors caninclude, without limitation, one or more accelerometer(s), gyroscope(s),camera(s), magnetometer(s), altimeter(s), microphone(s), proximitysensor(s), light sensor(s), and the like. For instance, image datareferred to above may be captured using a camera, which may be aninstance of sensor(s) 1740. Additionally or alternatively, sensor(s)1740 may include an always-on vision sensor capable performing objectdetection in accordance with various implementations and embodimentsdescribed herein using an integrated camera or using image data receivedby the always-on vision sensor from a non-integrated camera sensor. Suchan always-on vision sensor may include various aspects described above.

Embodiments of the device may also include a Satellite PositioningSystem (SPS) receiver 1780 capable of receiving signals 1784 from one ormore SPS satellites using an SPS antenna 1782. The SPS receiver 1780 canextract a position of the device, using conventional techniques, fromsatellites of an SPS system, such as a global navigation satellitesystem (GNSS) (e.g., Global Positioning System (GPS)), Galileo, Glonass,Compass, Quasi-Zenith Satellite System (QZSS) over Japan, IndianRegional Navigational Satellite System (IRNSS) over India, Beidou overChina, and/or the like. Moreover, the SPS receiver 1780 can be usedvarious augmentation systems (e.g., an Satellite Based AugmentationSystem (SBAS)) that may be associated with or otherwise enabled for usewith one or more global and/or regional navigation satellite systems. Byway of example but not limitation, an SBAS may include an augmentationsystem(s) that provides integrity information, differential corrections,etc., such as, e.g., Wide Area Augmentation System (WAAS), EuropeanGeostationary Navigation Overlay Service (EGNOS), Multi-functionalSatellite Augmentation System (MSAS), GPS Aided Geo Augmented Navigationor GPS and Geo Augmented Navigation system (GAGAN), and/or the like.Thus, as used herein an SPS may include any combination of one or moreglobal and/or regional navigation satellite systems and/or augmentationsystems, and SPS signals may include SPS, SPS-like, and/or other signalsassociated with such one or more SPS.

The device 1700 may further include and/or be in communication with amemory 1760. The memory 1760 may incorporate memory and/or registerssuch as memory/register(s) 212 and 216 shown in FIG. 2, as well asmemory/register(s) 312 of FIG. 3. Furthermore, memory 1760 mayincorporate stage databases such as stage database 218 shown in FIG. 2and stage database 318 shown in FIG. 3. The memory 1760 can include,without limitation, local and/or network accessible storage, a diskdrive, a drive array, an optical storage device, a solid-state storagedevice, such as a random access memory (“RAM”), and/or a read-onlymemory (“ROM”), which can be programmable, flash-updateable, and/or thelike. Such storage devices may be configured to implement anyappropriate data stores, including without limitation, various filesystems, database structures, and/or the like.

The memory 1760 of the device 1700 also can comprise software elements(not shown), including an operating system, device drivers, executablelibraries, and/or other code embedded in a computer-readable medium,such as one or more application programs, which may comprise computerprograms provided by various embodiments, and/or may be designed toimplement methods, and/or configure systems, provided by otherembodiments, as described herein. In an aspect, then, such code and/orinstructions can be used to configure and/or adapt a general purposecomputer (or other device) to perform one or more operations inaccordance with the described methods.

It will be apparent to those skilled in the art that substantialvariations may be made in accordance with specific requirements. Forexample, customized hardware might also be used, and/or particularelements might be implemented in hardware, software (including portablesoftware, such as applets, etc.), or both. Further, connection to othercomputing devices such as network input/output devices may be employed.

With reference to the appended figures, components that can includememory can include a non-transitory machine-readable medium storinginstructions for performing the methods of FIGS. 14A, 14B, 15, and/or16. The term “machine-readable medium” and “computer-readable medium” asused herein, refer to any storage medium that participates in providingdata that causes a machine to operate in a specific fashion. Inembodiments provided hereinabove, various machine-readable media mightbe involved in providing instructions/code to processing units and/orother device(s) for execution. Additionally or alternatively, themachine-readable media might be used to store and/or carry suchinstructions/code. In many implementations, a computer-readable mediumis a physical and/or tangible storage medium. Such a medium may takemany forms, including but not limited to, non-volatile media, volatilemedia, and transmission media. Common forms of computer-readable mediainclude, for example, magnetic and/or optical media, punchcards,papertape, any other physical medium with patterns of holes, a RAM, aPROM, EPROM, a FLASH-EPROM, any other memory chip or cartridge, acarrier wave as described hereinafter, or any other medium from which acomputer can read instructions and/or code.

The methods, systems, and devices discussed herein are examples. Variousembodiments may omit, substitute, or add various procedures orcomponents as appropriate. For instance, features described with respectto certain embodiments may be combined in various other embodiments.Different aspects and elements of the embodiments may be combined in asimilar manner. The various components of the figures provided hereincan be embodied in hardware and/or software. Also, technology evolvesand, thus, many of the elements are examples that do not limit the scopeof the disclosure to those specific examples.

It has proven convenient at times, principally for reasons of commonusage, to refer to such signals as bits, information, values, elements,symbols, characters, variables, terms, numbers, numerals, or the like.It should be understood, however, that all of these or similar terms areto be associated with appropriate physical quantities and are merelyconvenient labels. Unless specifically stated otherwise, as is apparentfrom the discussion above, it is appreciated that throughout thisSpecification discussions utilizing terms such as “processing,”“computing,” “calculating,” “determining,” “ascertaining,”“identifying,” “associating,” “measuring,” “performing,” or the likerefer to actions or processes of a specific apparatus, such as a specialpurpose computer or a similar special purpose electronic computingdevice. In the context of this Specification, therefore, a specialpurpose computer or a similar special purpose electronic computingdevice is capable of manipulating or transforming signals, typicallyrepresented as physical electronic, electrical, or magnetic quantitieswithin memories, registers, or other information storage devices,transmission devices, or display devices of the special purpose computeror similar special purpose electronic computing device.

Terms, “and” and “or” as used herein, may include a variety of meaningsthat also is expected to depend at least in part upon the context inwhich such terms are used. Typically, “or” if used to associate a list,such as A, B, or C, is intended to mean A, B, and C, here used in theinclusive sense, as well as A, B, or C, here used in the exclusivesense. In addition, the term “one or more” as used herein may be used todescribe any feature, structure, or characteristic in the singular ormay be used to describe some combination of features, structures, orcharacteristics. However, it should be noted that this is merely anillustrative example and claimed subject matter is not limited to thisexample. Furthermore, the term “at least one of” if used to associate alist, such as A, B, or C, can be interpreted to mean any combination ofA, B, and/or C, such as A, AB, AA, AAB, AABBCCC, etc.

Having described several embodiments, various modifications, alternativeconstructions, and equivalents may be used without departing from thespirit of the disclosure. For example, the above elements may merely bea component of a larger system, wherein other rules may take precedenceover or otherwise modify the application of the invention. Also, anumber of steps may be undertaken before, during, or after the aboveelements are considered. Accordingly, the above description does notlimit the scope of the disclosure.

What is claimed is:
 1. A hardware-based classification apparatuscomprising: a hardware engine configurable to implement a plurality ofcascade classifiers comprising at least a first cascade classifier and asecond cascade classifier, the hardware engine configurable to implementone or more stages within each of the plurality of cascade classifiers,wherein the first cascade classifier and the second cascade classifierare configured to perform classification based on image data stored in acommon image data buffer, the first cascade classifier is configured toperform a classification different from a classification performed bythe second cascade classifier, and the hardware engine is capable ofconditionally branching from the first cascade classifier to the secondcascade classifier based on an outcome condition associated withevaluating of at least one stage of the first cascade classifier.
 2. Theapparatus of claim 1, wherein the image data stored in the common imagedata buffer comprises image data representing a portion of an image, aportion of an integral image, or an integral image of a portion of theimage.
 3. The apparatus of claim 2, wherein the first cascade classifieris configured to perform object detection on a first window size of theimage data and the second cascade classifier is configured to performobject detection on a second window size of the image data differentfrom the first window size.
 4. The apparatus of claim 1, wherein thefirst cascade classifier is configured to perform object detection of afirst object class and the second cascade classifier is configured toperform object detection of a second object class different from thefirst object class.
 5. The apparatus of claim 1, wherein the outcomecondition comprises a branch-on-fail condition corresponding to afailure by the at least one stage of the first cascade classifier todetect an object within a window of the image data stored in the commonimage data buffer.
 6. The apparatus of claim 1, wherein each stagewithin the each of the plurality of cascade classifiers corresponds to ahardware stage storing stage data to enable the hardware engine toimplement the each stage, wherein each hardware stage is associated witha unique hardware stage identifier.
 7. The apparatus of claim 6, whereinhardware stages implementing logical stages of the first cascadeclassifier and similar logical stages of the second cascade classifierare assigned hardware stage identifiers within a common range of values.8. The apparatus of claim 6, wherein the apparatus is configured torecord a value representing a highest hardware stage identifier reachedduring an execution of the hardware engine.
 9. The apparatus of claim 1,wherein the hardware engine has access to memory to enable storing ofstage-related data usable by the hardware engine of the plurality ofcascades.
 10. The apparatus of claim 1, wherein each stage within thefirst cascade classifier and the second cascade classifier employs amodular design having similar hardware capabilities as other stageswithin the first cascade classifier and second cascade classifier.
 11. Amethod for performing classification comprising: performingclassification with a first cascade classifier based on image datastored in a common image data buffer; and performing classification witha second cascade classifier based on the image data stored in the commonimage data buffer, wherein the classification performed by the secondcascade classifier is different from the classification performed by thefirst cascade classifier, wherein performing the classification with thefirst cascade classifier comprises: determining an outcome conditionassociated with evaluation of at least one stage of one or more stagesof the first cascade classifier, responsive to the outcome conditionmatching a pass condition, proceeding to a next stage of the one or morestages of the first cascade classifier, and responsive to the outcomecondition matching a fail condition, conditionally branching from thefirst cascade classifier to a stage of one or more stages of the secondcascade classifier.
 12. The method of claim 11, wherein the image datastored in the common image data buffer comprises image data representinga portion of an image, a portion of an integral image, or an integralimage of a portion of the image.
 13. The method of claim 12, whereinperforming the classification with the first cascade classifiercomprises performing object detection on a first window size of theimage data, and performing the classification with the second cascadeclassifier comprises performing object detection on a second window sizeof the image data different from the first window size.
 14. The methodof claim 11, wherein performing the classification with the firstcascade classifier comprises performing object detection of a firstobject class, and performing the classification with the second cascadeclassifier comprises performing object detection of a second objectclass different from the first object class.
 15. The method of claim 11,wherein the outcome condition comprises a branch-on-fail conditioncorresponding to a failure by the at least one stage of the firstcascade classifier to detect an object within a window of the image datastored in the common image data buffer.
 16. The method of claim 11,wherein each stage within the each of the first cascade classifier andsecond cascade classifier corresponds to a hardware stage storing stagedata to enable the hardware engine to implement the each stage, whereineach hardware stage is associated with a unique hardware stageidentifier.
 17. The method of claim 16, wherein hardware stagesimplementing logical stages of the first cascade classifier and similarlogical stages of the second cascade classifier are assigned hardwarestage identifiers within a common range of values.
 18. The method ofclaim 16, further comprising recording a value representing a highesthardware stage identifier reached during an execution of the firstcascade classifier and the second cascade classifier.
 19. The method ofclaim 11, wherein the hardware engine has access to memory to enablestoring of stage-related data usable by the hardware engine acrossdifferent cascades of a plurality of cascades.
 20. The method of claim11, wherein each stage within the first cascade classifier and thesecond cascade classifier employs a modular design having similarhardware capabilities as other stages within the first cascadeclassifier and second cascade classifier.
 21. A system for performingclassification comprising: means for performing classification with afirst cascade classifier based on image data stored in a common imagedata buffer; and means for performing classification with a secondcascade classifier based on the image data stored in the common imagedata buffer, wherein the classification performed by the second cascadeclassifier is different from the classification performed by the firstcascade classifier, wherein the means for performing the classificationwith the first cascade classifier comprises: means for determining anoutcome condition associated with evaluation of at least one stage ofone or more stages of the first cascade classifier, means for,responsive to the outcome condition matching a pass condition,proceeding to a next stage of the one or more stages of the firstcascade classifier, and means for, responsive to the outcome conditionmatching a fail condition, conditionally branching from the firstcascade classifier to a stage of one or more stages of the secondcascade classifier.
 22. The system of claim 21, wherein the image datastored in the common image data buffer comprises image data representinga portion of an image, a portion of an integral image, or an integralimage of a portion of the image.
 23. The system of claim 22, wherein themeans for performing the classification with the first cascadeclassifier comprises means for performing object detection on a firstwindow size of the image data, and the means for performing theclassification with the second cascade classifier comprises means forperforming object detection on a second window size of the image datadifferent from the first window size.
 24. The system of claim 21,wherein the means for performing the classification with the firstcascade classifier comprises means for performing object detection of afirst object class, and the means for performing the classification withthe second cascade classifier comprises means for performing objectdetection of a second object class different from the first objectclass.
 25. The system of claim 21, wherein the outcome conditioncomprises a branch-on-fail condition corresponding to a failure by theat least one stage of the first cascade classifier to detect an objectwithin a window of the image data stored in the common image databuffer.
 26. The system of claim 21, wherein each stage within the eachof the first cascade classifier and the second cascade classifiercorresponds to a hardware stage storing stage data to enableimplementing the each stage, wherein each hardware stage is associatedwith a unique hardware stage identifier.
 27. The system of claim 26,wherein hardware stages implementing logical stages of the first cascadeclassifier and similar logical stages of the second cascade classifierare assigned hardware stage identifiers within a common range of values.28. The system of claim 26, further comprising means for recording avalue representing a highest hardware stage identifier reached during anexecution of the first cascade classifier and the second cascadeclassifier.
 29. The system of claim 21, wherein each stage within thefirst cascade classifier and the second cascade classifier employs amodular design having similar hardware capabilities as other stageswithin the first cascade classifier and second cascade classifier.
 30. Anon-transitory computer-readable medium storing instructions therein forexecution by one or more processing units, comprising instructions to:perform classification with a first cascade classifier based on imagedata stored in a common image data buffer; and perform classificationwith a second cascade classifier based on the image data stored in thecommon image data buffer, wherein the classification performed by thesecond cascade classifier is different from the classification performedby the first cascade classifier, wherein the instructions to perform theclassification with the first cascade classifier comprises instructionsto: determine an outcome condition associated with evaluation of atleast one stage of one or more stages of the first cascade classifier,responsive to the outcome condition matching a pass condition, proceedto a next stage of the one or more stages of the first cascadeclassifier, and responsive to the outcome condition matching a failcondition, conditionally branch from the first cascade classifier to astage of one or more stages of the second cascade classifier.